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MEASURES  OP  SYNTACTIC  COMPLEXITY 


Y.  Bar-Hillel,  A«  Kasher  and  E«  Shamir 


Section  0«  Introduction 

The  aim  of  this  report  is  twofold:  1)  to  establish  certain  formal 
connections  between  three  explicata  for  the  concept  of  syntactic  complexity, 
that  have  been  discussed  in  rec'ent  literature:  degree  of  nesting,  degree  of  self- 
embedding^  and  depth  of  postponed  symbols,  2)  to  present  a  formal  (and  trivial) 
proof  of  what  has  been  called  elsewhere  [2,  p.  13]  the  Anti-Wittgensteinian  Thesis: 
Rot  everything  that  can  be  said  at  all  can  be  said  by  using  syntactically  simple 
sentences  exclusively.  This  thesis  is  shown  to  hold  for  all  standard  propositional 
logics,  under  the  assumption  that  they  contain  means  of  expressing  infinitely 
many  different  sentences  and  under  all  plausible  conceptions  of  semantic  equivalence 
it  is  also  proved  true  for  a  certain  calculus  of  arithmetical  funotions« 


Section  1.  Degree  of  nesting  and  depth  of  postponed  symbols 

We  shall  follow,  with  minor  modifications,  the  notations  and  terminology 

of  [3], 

V  is  a  given  set  -  the  vocabulary.  Elements  of  V  will  be  called 
symbols  and  denoted  by  capital  Latin  letters.  Finite  sequences  of  symbols  of  V  - 

including  the  empty  sequence  -  will  be  called  strings  over  V  and  denoted  by 

small  Latin  letters.  Sets  of  strings  over  V  wiii  be  called  languages  over  V 

and  denoted  by  L  with  subscripts,  in  general.  If  L,  we  say  that  i  is  a 

sentence  of  L.  The  set  of  all  strings  over  V  will  be  denoted  by  W^  .  The 


length  of  the  string  x  is  the  number  of  its  symbol  occurrences. 
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By  a  grammar  we  understand  a  finite  system  of  rules  determining  a  language# 
Grammars  will  be  denoted  by  Gothic  capitals# 

Definition  1,1  (a)  A  context-free  grammar  (CFG)*  is  an  ordered  q[uadruple 
=  (V,  P,  T,  S),  where 

(i)  V  is  a  finite  vocabulary; 

(ii)  P  is  a  finite  set  of  productions  of  the  form  X  x  where  XeV, 
xeW^,  X  ^  Xj 

(iii)  T  is  a  subset  of  V  (the  terminal  vocabulary),  none  of  whose 
elements  occur  on  the  left  side  of  a  production  of  P; 

(iv)  S'^  (the  initial  symbol)  is  a  distinguished  element  of  V  -  T  (the 
auxiliary  vocabulary) . 

(b)  y  directly  generates  z  (y  z),  iff  y  =  uXv, 
z  =  UXV  and  X  xe  P„ 


(o)  y  generates  z  (y  =%>  z),  iff  there  exists  a  seq.uence' 
of  strings  z^,z^, , , .  ,z^  (r>0),  such  that 

y  =  z^,  z^  =  z  and  z^  (i  =  1 , , » .  ,r) , 

The  seq[uence  z^,o,,,z^  will  be  called  a  generation  tree  of  z  from  y„ 

(d)  X  is  a  sentence  generated  by  iff  x  is  a  string 

over  T,  and  S  ^x,  L((^)  is  the  set  of  all  sentences  generated  by  „ 

(e)  A  language  is  representable  by  a  CFG  ,  or  is  a 

context-free  language  (CFL),  iff  there  exists  a  CFG  such  that  L  -  L((^), 


*Also  called  a  simple  phrase-structure  grammar  (SPG). 
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Let  us  represent  direct 
the  following  schema: 


generation  of  the  form  uAv=^  hy 


u  A  V 

B  B>  0  o  oB, 
oik 

Then  any  generation  X  x  may  he  represented  graphically  hy  a  labelled 
generation-tree  (alternatively:  tree,  phrase  marker  [4])j  cfo  Fig  1.  Such 
a  tree  is  essentially  a  branching  pattern,  consisting  of  nodes  each  of  which  is 
labelled  hy  a  symbol  of  V  »  Referring  to  the  direct  generation  given  above,  each 
node  labelled  B.  (1$  j<k)  is  said  to  follow  the  node  labelled  A  «  In  any 
tree  there  is  one  node,  and  only  one,  which  does  not  follow  any  other;  this 
node  is  called  the  root  of  the  tree.  A  terminal  node  is  one  which  has  no 
followers. 

For  certain  purposes  it  is  useful  to  introduce  a  special  notation  for  the 
nodes.  Denote  the  root  by  0,  and  its  followers,  from  left  to  right  (or, 
conversely,  from  right  to  left  -  we  shall  have  occasion  to  use  this  converse 
notation)  by  00,  01,..., Qk,  eto.  In  general,  if  a  node  with  k+1  followers 
is  denoted  by  a  multi-index  0(  ,  then  its  followers,  from  left  to  right 
[from  right  to  left],  are  denoted  by  OCO,  (Xl,...,6(k. 

If  we  order  the  multi- indices  lexicographically,  we  thereby  induce  an 
ordering  of  the  nodes  (of.  Fig.  2).  It  i&- clear  that  in  order  to  obtain  the 
string  I  Vhose  generation  is  represented  by  the  tree,  we  must  concatenate 
the  symbols  labelleing  the  terminal  nodes,  according  to  the  above  order. 

A  path  in  a  tree  is  a  sequence  of  nodes,  beginning  at  the  root,  such  that 
each  node  (except  the  root)  follows  its  predecessor  in  the  sequence.  A  path 
is  complete  if  its  last  node  is  terminal  (i.e., 


it  leads  to  a  terminal  node). 
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Figure  1  Figure  2 

The  productions  taking  part  are:  (The  same  tree  as  in  Figure  1) 

S  ^  ABCj  A  — ^  BDj  B  — ^  EDj 
C  BDCj  C  E« 

(The  string  generated  is  EDBEDBDDEDBE) 


(a)  (h) 

Figure  3 

(h)  shows  that  the  depth  of  left-postponed  symbols  may  be  made  arbitrarily 
large.  (A  similar  figure  gives  the  result  for  ^  ) 


-  4  - 


Clearly,  each  [terminal^  node  in  a  tree  uniquely  determines  a  [complete]  path  - 
ioSo,  the  path  leading  to  ito 

The  two  extreme  (ioe.,  the  leftmost  and  the  rightmost)  terminal  nodes  and 
all  the  nodes  on  the  paths  leading  to  them  are  called  the  boundary  nodes  of 
the  tree®  All  the  others  are  called  inner  nodes. 

Each  node  N  in  a  tree  determines  in  a  natural  way  a  subtree  rooted  in  E. 
This  subtree,  when  considered  in  isolation  from  the  nest  of  the  original  tree, 
is  itself  a  tree;  the  relativization  of  the  above  concepts  to  the  subtree 
is  self-evident. 

Returning  to  the  multi-index  notation,  the  absolute  value  of  a  multi¬ 
index  <x  (and  of  the  corresponding  node)  is  defined  to  be  the  sum  of  the 
indices  in  (n  .  Omitting  from  a  multi- index  oc  all  indices  except  the  last, 
the  absolute  value  of  a  node  N  is  then  the  sum  of  all  the  indices  of  the 
nodes  on  the  path  leading  to  N.  This  last  notation  (i.e.,  using  only  indices) 
was  that  used  by  Yngve  [l4j.  Following  his  lead^as  elaborated  by  Chomsky  [7], 
we  define 

Definition  1.2.  Let  P  be  a  tree;  then  the  depth  of  left-postponed  symbols 
of  P  ,  P)  >  is  the  maximum  absolute  value  of  its  nodes»  The  depth  of 
right-postponed  symbols^  defined  in  a  similar  manner,  using  the 

converse  ravilti-index  notation  (i.e.,  counting  the  nodes  from  right  to  left). 

The  significance  of  these  concepts  will  be  discussed  only  after  the  introduction 
of  two  other  concepts,  due  to  Chomsky  [6,7]]>  and  after  the  establishment  of 
some  of  their  basic  properties. 

Definition  1.3o  A  node  of  a  tree  is  nesting  [self-embedding  (SE)J  iff 
the  subtree  rooted  therein  contains  a  non-terminal  [similarly  labelled]  inner 


node  (with  respect  to  that  same  subtree),,  The  two  nodes  form  a  nested 
[self-embedded]  pair® 

Definition  lo4o  (a)  The  degree  of  nesting  „  V  ( P) j  of  a  (terminal) 
tree  P  is  the  largest  integer  m  having  the  following  property: 

Th'^re  is  a  complete  path  in  P  through  m+1  nodesy  y  o .  o  where 

each  (l^i^m)  is  an  inner  node  in  the  subtree  rooted  in 

(b)  If  the  nodes  (l$i$lm)  are  all  similarly 

labelled,  the  integer  thus  defined  is  called  the  degree  of  self~ embedding, 

£(r)«^ 

Theorem  l.l.  The  following  inegualities  hold; 

(a)  Sir)  ^AP) 

(15)  v(r)  ^^(r),  y(r)^,^(r), 

(c)  ^  H  is  the  number  of  auxiliary  symbols  in  the  grammar, 

then 


e 


( [x]  denotes  the  integral  part  of  i) « 


Proof.  (a)  and  (b)  are  obvious. 

To  prove  (c),  note  that  if  ^(P)  ^  kM,  then  there  are  at 
least  kM+1  nodes  which  satisfy  the  condition  of  Def.  1.4.  (a),  and  all 


%ote  that  this  definition  differs  slightly  from  that  given  by  Chomsky  [6,  p.  llj, 
because  of  the  inequality  in  (iv)  of  his  definition.  This  fact  may  change 
slightly  a  number  of  results. 
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of  them  except  the  last  are  labelled  by  auxiliary  symbols^  Thus,  at  least  k 
are  similarly  labelled,  so  that  6  (T  )  ^  k  =  f  1 » 


Since  (c)  implies  y  (  P )  -S'  M(i  (  P  )  +1),  it  may  be  said  that,  for  a 
given  grammar  G  ,  ^((~' )  and  L(f")  are,  up  to  a  certain  multiplicative 

constant,  equivalent  measureso  This  is  not  true  of  y  (r  )  (or  £  (P  ))  and 
the  depths  ^(^)s  ^'or  we  may  have  v'(P)  =  1>  while  both 

and  }(  P)  are  arbitrarily  large  (cf.  Pig,  3)»  Clearly,  the  two  depths 
)  and  ^  ( P }  are  also  incomparable,, 

Nevertheless,  the  following  result  (given  without  proof)  shows  that  a 
connection  of  sorts  exists  between  fi  and  £  : 


Theorem  lo2<i  For  every  CF-grammar  there  exist  two  constants,  and 

b^  [a^  and  b^  ],  depending  only  upon  the  grammar,  such  that  for  every 
generation  tree  P  of  the  grammar,  the  following  inequality  holds; 


j(r)  ^a^e(P)  +  [^('^)^  a^£(r)  ]. 

(Equality  may  be  realized  in  certain  trees.). 

Consider  now  a  tree  one  of  whose  terminal  nodes,  N  ,  is  "replaced" 

by  another  tree  (N  being  relabelled  by  the  label  of  the  root  of  f^). 

Call  the  resulting  composed  tree  P  (cfo  Fig„  4)o 


Theorem  lo3.  The  following  inequalities  hold; 

(a)  j(P)^  Mr).<  ;i(q)+MQ) 

(b)  y(r)  ^  y(r^)  +v(Q)  +  1 

s(r)  ^  £(q)  + 1 
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Moreover i)  if  N  is  on  a  path  which  determines  the  respective  depth  or  decree 
of  s  then  equality  holds  in  (a.)  and  either  equality  holds  in"('b)  or 

y(r)  -  )-(q) 

Proof 0  All  the  assertions  are  ohviouso  It  is  only  necessary  to  e^cplain 
why  y(r)  (or  £;(P))  may  be  larger  by  one  than  the  sumo  This  may  occur 
if  there  are  three  nodes s  M9  N  and-  P  j  on  the  same  path  in  P  j  such 
that  neither  (MjM)  nor  (NjP)  is  a  nesting  pair,  but  (M,P)  is  (cfo  Pigo  5)» 

Note  that  the  additivity  of  the  second  part  of  i-ne  theorem  is  in  general 
not  true  for  £'(r’)j  since  the  labels  of  the  longest  SS  sequences  in  q 
and  q  may  be  different. 

The  concepts  ^  ^  S  ^  introduced  above  as  measures  of  the 

complexity  of  trees  emerged  in  the  work  of  Yngve  [14]  and  Chomsky  [6,7] ,  who 
were  engaged  in  the  construction  of  models  to  explain  the  linguistic  behavior 
of  users  of  natural  languages.  A  detailed  discussion  and  comparison  of  both 
approaches  and  an  explanation  of  the  way  they  arise  appears  in  Chomsky  [7] 

(cfo  also  Bar-Hillel  [pj  Second  LectureJ).  Here  we  shall  briefly  summarize 
some  basic  points. 

Both  authors  assume  that  the  model  should  be  presented  as  a  strictly 
finite  device  ^  9  which  is  usually  called  a  finite  automaton  with  output 
(or  a  finite  transducer) .  This  device  operates  on  the  sentences  of  the  language 
(as  either  inputs  or  outputs)  from  left  to  right.  Moreover9  it  is  assumed  that 
the  "permanent  memory"  of  ^  contains  a  CF-grammar  of  the  language. 

Now  in  the  Yngve  model  (which  was  originally  intended  as  a  description  of 


the  speaker's  behavior)  the  device  ^  ,  in  order  to  generate  a  sentence, 
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constructs  generation  trees  from  root  to  terminal  nodes  and  from  left  to  right* 
This  implies  that  whenever  a  direct  generation  A  B  oooB  is  reachedy 

*  0  iC 

in  which  B.  is  the  leftmost  non-terminal,  then  the  terminals  B  ,.oo,B.  . 

J  o’  j-1 

are  added  to  the  output,  whereas  ®  ^  transferred  to  the  temporary 

memory  and  the  Branching  of  is  postponed  till  the  subtree  rooted  in  B^ 

is  fully  generated. 

Now,  it  is  easily  verified  that  ^(P),  the  depth  of  right-postponed 
symbols,  is  the  maiimum  number  of  symbols  in  the  temporary  memory  during  the 
generation  of  P  ,  In  other  words,  if  the  capacity  of  the  temporary  memory 
is  fixed  and  equal  to  k  ,  then  ^  can  handle  only  trees  such  that 

If  we  assume  that  ^  deals  with  trees  in  the  opposite  direction,  from 
terminals  to  root  (but  again  from  left  to  right),  then  the  role  of  ^ 
will  be  played  by  A(r),  the  depth  of  left-postponed  symbols. 

Note  that  for  ^(P)  to  remain  bounded,  only  a  limited  number  of  symbols 
which  are  not  rightmost  may  branch}  on  the  other  hand,  rightmost  symbols  may 
branch  arbitrarily  often.  Thus  the  tree  is,  of  necessity,  predominantly 
right-recursive.  Similarly,  trees  for  which  >(r)  is  bounded  are  predominantly 
left-recursive. 

Suppose  now  that  no  restrictions  are  imposed  upon  the  manner  in  which  <£) 
operates  on  trees.  Then  the  degree  of  nesting  emerges  as  a  natural  measure. 

Indeed,  the  following  assertion  results  from  Chomsky's  work  [5]: 

It  is  possible  to  construct  a  device  and  a  strictly  monotonic  function 

g^ ( m) ,  such  that,  if  the  available  computing  space  of  m  ,  then  can 

handle  those,  and  only  those,  trees  P  (of  a  grammar  0  )  for  which 
V'(r’)  $  g^(ni). 


In  view  of  the  equivalence  of  the  measures  >'(r’)  and  £(r")(,  the  same 
result  (with  a  different  functions,  §2^”^^  holds  for  indeed, 

Chomsky's  results  are  usually  formulated  for  ^  (r')c 

A  critique  of  Yngve's  more  restrictive  assumptions,  which  lead  to  _f(^) 
or  ^(r),  is  given  in  Chomsky  ["y]"  But,  even  for  unlahelled  trees,  there 
is  no  reason  why  right  or  left-recursion  should  he  singled  out,  and  the  degree 
of  nesting  seems  to  us  a  more  adequate  measure  of  complexityo 

¥e  add  a  few  remarks  explaining  why  the  degree  of  nesting  is  in  a  sense 
a  more  natural  measure  than  the  degree  of  self-emhedding<, 

The  degree  of  nesting  (like  the  two  depths)  depends  only  on  the  tree  and 
not  on  the  labelling.  This  is  not  so  in  the  case  of  the  degree  of  SE.  For 
this  reason,  v(r)  compares  better  with  (^)?  which  are 

themselves  useful  in  many  cases.  This  accounts  also  for  the  lack  of  additivity 
for  lir)  in  Theorem  1.3.  This  kind  of  additivity  seems,  however,  to  be  a 
natural  formal  requirement  for  the  adequacy  of  a  proposed  measure  of  complexity. 

Another  consequence  of  this  dependence  on  labelling  is  that  £(r')  is 
very  unstable,  even  for  slight  changes  in  the  grammar.  For  instance,  consider 
a  tree  ^  for  which  E.  ( P )  =  k  and  A  is  the  only  SE  symbol.  Let 
A  =  A^  and  add  to  the  grammar  a  replica  A^  of  A.  Assuming,  for  the  sake 
of  convenience,  that  k  is  even,  replace  every  second  occurrence  of  A  on 
every  path  by  A^.  Clearly,  5  is  decreased  thereby  to  —  ,  while  is 

not  changed.  In  a  similar  manner,  ^  may  be  decreased  m-fold,  for  any 
natural  m  ,  by  taking  ra  replicas  of  A, 

To  summarize,  it  seems  rather  clear  that  nesting  is  the  main  phenomenon 
responsible  for  syntactic  complexity  of  sentences.  It  is  nesting  which 
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creates  a  dependence  "between  the  parts  to  the  left  and  to  the  right  of  the 
nested  elements 

Section  2o  Indispensahility  of  arbitrarily  complex  sentences  in  propositional 
logics 

Our  aim  in  this  section  is  to. show  that,  in  general,  unbounded  complexity 
of  sentences  is  essentially  indispensablOo  To  this  purpose,  we  shall  prove 
that  certain  very  elementary  propositional  logics  contain  sentences  of 
arbitrarily  high  degrees  of  nesting,  which  are  not  (semantically)  equivalent 
to  any  sentences  of  lower  degree#  This  result  also  holds,  albeit  with  certain 
restrictions,  for  the  other  measures  of  syntactic  complexity  discussed  in  §1. 

One  reason  for  our  interest  in  (various)  propositional  logics,  apart 
from  the  fact  that  their  formation  rules  are  easuly  expressed  as  CF-grammars, 
is  that  some  of  their  sentential  connectives  have  syntactically  close 
analogues  in  natural  languages}  this  is  not  true  to  the  same  extent  for  the 
additional  symbols  of,  say,'  quantification  logic#  Thus  our  results  are  of 
rather  direct  significance  for  natural  languages#  Indeed,  the  simplest  way 
(though  not  the  deepest)  of  exhibiting  complexity  and  various  other  phenomena 
in  natural  languages  has  often  been  by  recourse  to  tricks  of  propositional 
logic# 

■*^t  is  natural  to  assume  that  this  dependence  creates  a  load  on  memory  and 
concentration.  What'  additional  tensions,  if  any,  are  created  by  self- embedding,  in 
addition  to  those  ^.caused  by  nesting,  remains  to  be  investigated# 
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All  propositional  calculi  compared  here  will  contain  some  or  all  of  the 
following  generation  rules s 

(1)  S  FS 

(2)  S  [SGS]  or  (2>)  S  GSS 

(3)  S  »»  T 

(4)  T  T' 

(5)  T  P 

In  addition,  each  particular  calculus  will  contain  rules  req,uiring  that 
G  [and  in  the  case  of  Pi,  P3,  also  fJ  he  rewritten  as  a  binary  [unary] 
Gonneotiyeo  In  particular,  we  consider: 

Plo  (Propositional  calculus  in  Russell  notation):  (1)  -  (5)?  and: 

F  — ^  ^  ,  G  ,  G  V  ,  G  &  (negation,  implication,  disjunction 

and  conjunction,  respectively) » 

P2.  (Eq.uivalence  calculus):  (2)  -  (5)j  and;  G  — »  r  (equivalence) » 

The  corresponding  calculi  in  Polish  notation  are: 

(1),  (2')j  (3)  -  (5)  and:  F  U,  G  C,  G  A,  G  K„ 

P4o  (2'),  (3)  -  (5)  and:  G  E„ 

In  all  these  calculi,  rules  (3)  ~  (5)  are  used  (following  Curry)  to  form 

an  infinite  number  of  propositional  variables  (pv):  P,  P’,  P",  etco  It  will 

be  convenient  to  use  informally  P^  instead  of  P,  P^  instead  of  P',  etCo 

Remark:  In  the  usual  formulations  of  propositional  calculi,  one  assumes 

the  availability  of  infinitely  many  propositional  letters,  say  P^,  P^,  . 

In  this  way,  the  uninteresting  complexity  which  may  arise  from  the  construction 
of  P',  P",  etCo  is  avoidedc  In  fact,  the  rule  T  — ?  T'  is  left-recursive. 


- ---nnr  w7friT~raTn^|g«*aKgaM»»^ 
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so  that  its  repeated  use  increases  ^  <,  the  depth  of  right-postponed  symbolso 
However j  it  is  possible  to  use  instead  the  rule  T  'T,  which  is  right- 
recursive  and  so  increases  the  other  depthj  ^  „  If  it  is  desired,  to  avoid 
both  contingencies}  the  relevant  grammars  may  be  regard.ed  as  CF-grammars 
with  an  infinity  of  terminal  symbolSo  The  basic  properties  of  CF-grammars 
remain  unchanged  therebyo 

Note  that  any  generation  in  the  above  grammars  may  be  obtained  by  first 
using  rules  (i)  -  (2)  (or  (1)  -  (2'))  (in  Plj  P3)  to  construct  the  "skeleton" 
of  the  sentence  and  then  using  rules  (3)  -  (5)  and  the  substitutions  for  F 
and  G  to  differentiate  between  the  various  pv  and  the  oonnectivesc  It 
is  the  trees  of  the  "skeleton  grammar"  given  by  rules  (1)  -  (2)  (or  (l)  -  (2’)) 
which  will  essentially  determine  the  complexityo  (Cfo  the  remark  for  the  role 
of  rules  (3)  (5)  and  their  effect  op  the  depths,)  (Regarding  P2  and  P4j 

only  rule  (2)  (or  (2'))  is  used  in  the  skeleton  grammaro  This  is  to  be 
understood  in  the  sequel,) 

It  is  easily  established  that  the  grammars  Pi  -  P4  are  monotectonic, 

ioSo}  every  sentence  has  a  ixnique  derivation  tree.  Thus  there  is  no 

ambiguity  in  simply  speaking  of  the  degree  of  nesting  (or  depth)  of  a  sentence  i  , 

meaning  thereby  the  degree  of  nesting  (or  depth)  of  its  unique  derivation 

tree  /  »  This  would  no  longer  be  true  if  rul.e  (2)  were  replaced  by 

z 

(2")  S  SGS 

since  then  SGSGS  ,  for  example,  would  have  two  generation  trees  (cf.  Fig,  6), 

For  the  connection  between  monotectonicity  (ieSo,  syntactic  non-ambiguity) 
and  syntactic  complexity,  see  [2,  Second  LectureJ, 

Consider  a  string  x  generated  by  rules  (1)  -  (2),  Denote  by  s(i),  g(x), 
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f(x)  the  nximher  of  ocourrences  in  x  of  the  symbols  Sj  G,  Pj  respectivelyo 

Similar  notations  are  used  for  a  string  y  generated  by  rules  (1)  -  (2')o 
(in  the  following,  we  shall  adhere  to  the  convention  whereby  i  is  a  string 
generated  by  (1)  -  (2),  and  y  -  by  (1)  -  (2’)»)  Clearly,  s(z)  =  g(x)  +  1, 
and  ~  degrees  of  nesting  and  of  self-embedding  coincide, 

since  there  is  only  one  branching  symbol,  S. 

Lemma  2A<,  If  is  the  generation  tree  of  x  ,  then; 

(a)  y  (r^)  =  k  k^g(x)$2^-.l 

N  V(r)=k  =#  k5g(y) 

V 

and  every  value  of  g  permitted  by  these  inequalities  may  be  realized  effectively. 

Proof 0  The  proof  of  (a)  is  by  induction  on  ko 

The  case  k  =  1  is  trivial,  since  the  only  strings  x  for  which 
y(f^)  =  1  are  [SGsJ,  and  any  other  string  obtained  from  this  string  by 
use  of  rule  (l)  alone,  which  introduces  no  G'So 

Assume  the  theorem  true  fCr  a  given  k,  and  let  y  (f^)  =  k»  Then  y  may 
be  increased  to  k+1  in  a  variety  of 'ways;  the  two  extreme  cases  are  the 
following;  (1)  Rule  (2)  is  applied  to  a  single  S  on  a  path  determining 
the  degree  -  if  x'  is  the  resulting  string  then  g(i')  =  g(x)  +  lo 

(2)  Rule  (2)  is  applied  to  each  of  the  g(x)  +  1  cccurrences  of  S  in  x; 

each  application  introduces  a  new  G,  and  thus  g(x')  =  2g(x)  +  1^2(2^  -  1)  +  1  = 

=  2^"*^^  -  1  (by  the  induction  hypothesis) »  Obviously,  all  intermediate  values 
may  be  obtained  by  appropriate  variations^ 

As  for  (b)^  note  that  an  increase  in  g(y)  may  or  may  not  entail  an 


increase  in  V  o  Indeed^  in  S  -■?  GSS  GGSSS  the,  degree  of  nesting  increases 
at  the  second  stag's,  whereas  in  S  GS^=»>-GSGSS  there  is  no  increasso 

(The  hranching  symbol  is  underlined  in  each  cassa)  Thus  (b)  is  proved,  by 
a  trivial  inductiono  Note  that  no  upper  bound  (in  terms  of  V  )  is  possible 
for  g(y)  -  cf«  the  tree  in  Figo  7j  where  y  =  1  arxd  g  may  assume  any 
desired  value# 

Thus,  for  rules  (1)  -  (2),  s(^)  and  not  less 

than  log2(g(i)  +  l)o  No  similar  lower  bound  exists  for  (rules 

(1)  “  (2')) 5  while  it  may  assume  any  value  ^  g(y)o 

Similar  (though  slightly  more  complicated)  boiuids  may  bo  obtained  for 
y(r^)o  For  again,  only  an  upper  bound  is  obtained  (of#  Fig.  7, 

where  j  =  2  and  g(y)  is  not  bounded)#  In  the  computations  for  2  it  is 
necessary  to  take  f(x)  into  account,  since  the  value  of  ^  is  increased  by 
the  application  of  the  rule  S  — ^  FS# 

Passing  now  from  the  skeleton  grammars  to,  any  one  of  Pi  ~  P4j  b{x)  is 
the  number  of  pv  in  the  generated  sentence  x  ,  f(x)  the  number  of  unary 
and  y(x)  the  number  of  binary  connectives#  By  Theorem  lo3  (a),  the  depth 
is  not  increased  by  application  of  the  rule  T  — >T*,  andj>  -  by  application 
of  T  'T  (cfo  the  remark  on  page  11  ). 

Ne  shall  now  turn  to  the  semantics  of  the  propositional  logics#  First, 
consider  the  sentences  of  PI  as  truth  functions;  i»e#,  assign  to  each  pv 

the  "truth-value"  1  or  0  (true  or  false) ,  and  define  the  functions 
•^P^>  P^  ^  P2>  ^  P2’  % ^2’  ^1“  ^2^  whose  values  are  again  1  or 

0,  in  the  usual  manner  (For  instance,  val(p^  P^)  =  0  if  val(p^)  =  1 
and  val(p2)  =  Oj  otherwise  val(p^O  p^)  =  !•)  Two  sentences  are  termed 
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equivalent  iff  they  correspond  to  identical  truth“functionSo 

Denote  hy  q^  the  sentence  [Pq  r?  [P^  ^  °°°  ^  &n  1 ^ J  °°° 


Theorem  2 Ac  q^  is  not  equivalent  to  any  sentence  of  Pi  which  contains 
less  than  n+1  occurrences  of  pv  .  ^e  same  is  true  of  any  sentence  x 
obtained  from  q^  by  rebra.cketing» 

Proof o  It  is  easy  to  see  that  val(q^)  =  0  iff  val(p^)  =  val(p^)  -  « « » 

=  val(p^  1^  ~  ^  ^  change  in  the  value  of  any  one  variable 

changes  the  value  of  the  sentence  and  renders  it  true^  If  y  is  a  sentence 
containing  less  than  n+1  occurrences  of  pv  ,  then  one  of  the  variables  of 
q^  does  not  occur  therein,  and  a  change  in  the  value  of  this  variable  alone 
cannot  change  val(y)»  Hence  q^  is  not  equivalent  to  y  . 

The  same  argument  proves  the  assertion  not  only  for  q^,  but  also  for 
any  sentence  2  containing  n+1  pv  and  having  the  following  property: 

P'or  any  variable  P  in  x  there  exists  an  assignment  of  truth- valuer  to  the 
remaining  n  variables  such  that  a  change  in  val(p)  will  change  val(x) 

(the  remaining  n  truth- values  remaining  fixed). 

It  will  now  be  proved,  by  induction  on  n  ,  that  any  sentence  x  obtained 
from  q^  by  rebracketing  has  this  property.  The  assertion  is  trivial  for  n  =  2. 
In  general,  x  has  the  form  [yr)z].  If  the  chosen  variable  p  occurs  in  z, 
choose  an  assignment  of  truth-values  in  y  which  renders  y  true.  Then 
val([]y:?zj)  =  val(z)o  By  induction,  z  has  the  required  property,  and 
so  has  Xo  If  P  occurs  in  y  ,  choose  an  assignment  which  renders  z  false, 
and  then  val([y3z]])  =  1  -  val(y).  Again  by  induction,  y  has  the  required 
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propertys  and  so  has  x  o 

The  Theorem  is  thus  provedo 

Remark,.  A  calculus  is  called  m- valued  (m^2)  when  its  truth-fiuictions  are 
allowed  to  take  any  integer  truth- value  t  p  where  1  $t^mo 

For  the  negation-implication  m~valued  calculusj  truth-values  of  formulas 
are  determined  as  follows  [l2]s  (i)  val(^P)  =  nH-l-val(P) 

(ii)  val(P;3Q)  =  max(l jl+val(Q)  -  val(P))o 
The  results  of  this  chapter  hold,  with  some  slight  modifications,  for  these 
calculi  as  well,. 

It  is  well  kn.own  that  two  sentences  x  and  y  (in  PI)  are  equivalent 
iff  [[loy]  &  [js>x]]  is  derivable  from  an  appropriate  set  of  axioms  of  the 
propositional  oalculuso  Similarly,  P3  (Polish  notation),  x  and  y  are 
equivalent  iff  KCxyCyx  is  derivable  from  an  appropriate  set  of  axioms. 

Using  now  Lemma  2,1  and  Theorem  2,1 s 

Theorem  2,2,  For  every  natural  number  n  it  is  possible  to  construct  a  sentence 
X  i^  Pi  or  P3  such  that  v(  F^)  -  n  and  such  that  no  sentence  y  for 
which  ^  is  equivalent  to  x  ,  The  same  assertion  holds  for  and  ^ 

Proof.  For  Pi  ,  taice  x  =  q  „  Obviously,  v(r  )  =  n.  by  Theorem  2.1, 

^  '^n 

I  is  not  equivalent  to  any  sentence  containing  less  than  n+1  occurrences  of  pv 
and,  as  may  be  demonstrated  by  methods  of  the  propositional  calculus,  it  is 
not  equivalent  to  any  sentence  containing  more  than  n+1  pv  .  Moreover,  by 
similar  methods  it  is  easy  to  see  that  i  is  not  eq\iivalent  to  any  sentence 
obtained  from  itself  by  rebracketing  (which  may  have  a  smaller  degree  of  nesting). 
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The  assertion  is  thus  proved  for  Pi, 

For  P3  take  x  =  r'**'  9  where  s 

n 


r'**'  =  C  » , .  C  P  P.  . » 0 
n  >  0  1 


n 


P 

n 


(corresponding  to 

The  sentences  q  and  t  ,  or  their  counterparts 

n  n  ’ 

Polish  notation}  give  the  required  result  for  ^  and  o 


Pi  notation)  o 

q*  and  r*  in 
n  n 

j  respectively. 


The  same  results  regarding  the  degree  of  nesting  y  proved  using 

other  definitions  of  equivalent  sentences.  Indeed,  the  same  proof,  based  on 
Theorem  2.1,  gives  the  result  for  for  the  equivalence  calculus,  whose 
axioms  were  given  by  Lesniewski  [11,  P.  16]  (the  grammar  is  now  P2),  and  for 
the  same  in  Polish  notation  (grammar  P4). 


Other  possibilities  are: 

a)  X  is  equivalent  to  y  iff  =  y] ,  where  H  is  the  intuit ionistic 

H 

calculus  of  Heyting  [9]. 

b)  I  is  equivalent  to  y  iff  |-.  [2:  =  y]  >  where  E  is  the  entailment 

E 

calculus  of  Anderson-Belnap  (of.,  e.g. ,  [l]). 

c)  X  is  equivalent  to  y  iff  (En)  [I-  t=y]l»  where  P  is  the 

P  ^ 

n 

n-valued  logic  of  Lukasiewicz  (cf.,  e.g,,  [I2]). 

d)  X  is  equivalent  to  y  iff  f-  □[x  =  y],  where  S  is  any  modal 

S 


logic  of  Lewis  or  Fitch  [lO^  8]. 
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In  all  these  cases  (except  c)  the  result  follows  either  from  the  fact 
that  the  relevant  concept  of  equivalence  is  more  restricted  than  in  PI  with 
its  usual  axioms,  or  hy  direct  recourse  to  the  valuation  result  in  Theorem  2.1o 
(The  extension  of  this  theorem  to  multivalued  logic  proves  the  result  for  c. ) 

Note,  however,  the  following  result: 

Theorem  2o3t.  Every  sentence  x  in  the  equivalence  calculus  P2  is  equivalent 
to  a  sentence  y  for  which  =  It,  The  same  is  true  for  a  ,  and  for 

Polish  notation  (P4)» 

This  result  is  easily  derived  from  a  theorem  of  iukasiewicz: 

[p=  [asrJJ  =  r[prq.]  =  r]  (cfc  jl39  Po  4s»  footnote  2?  11,  po  I6j). 

Re marks 0  (1)  It  is  well  known  that  in  Russell’s  notation  any  sentence  is 

equivalent  to  a  sentence  containing  only  the  connectives  and 

or  ^  "  and  "v"o  However,  as  ia  easily  seen,  this  transformation  does  not 

reduce  the  degree  of  nestingo 

(2)  The  sentence  q^  is  equivalent  to 
[|Po  *  [Pl  =  pj  (also  according  to  definitions  h 

and  d)o  There  is  no  harm  in  omitting  brackets  and  writing 

FFp  &  P^  &oo<.&  P  hI-^P  I?  since  the  various  trees  yield  equivalent  sentenceso 
1 1-  0  1  n-l-*  n-* 

The  complexity  of  the  sentence  has  no  doubt  been  reduced;  however,  other 

sentences,  obtained  from  q  byrebracketing  (eogo,  the  sentence  r  , 

n  n 

[...[[p^3Pj]oP2]^...3PJ)  are  not  so  amisnable  to  simplif icationo 
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Section  3o  Complexity  of  arithmetical  functions 

In  this  seotiouj  measures  of  complexity  analogous  to  those  developed  in 
the  preceding  sections  for  sentences  will  he  developed  for  certain  functions 
over  the  natural  numherso  An  anti-Wittgensteinian  thesis  will  he  proved  for 
a  certain  representation  of  these  f-unctionso  It  must  he  emphasized  that  the 
concepts  to  he  introduced  are  measures  of  complexity  of  functions  in  a  certain^ 
well-defined,  sense  onlyo 

Define  a  CF-grammar  as  followss 

The  vocabulary  consists  of 

(i)  the  initial  symbol  Tj 

(ii)  a  finite  set  of  auxiliary  symbols  x.  where  i=l,„..,k  and, 
for  every  if  ^  =  1, ojn^? 

(iii)  the  terminal  symbols  f  , o..,f,  ,  y?  z.  .  (where  i  and  j  vary 

1'  K.  IfJ' 

as  in  (ii))j  P,  (,  and  )  „ 

NoB.  The  indices  i  and  j  above  are  not  meant  to  be  part  of  the 
relevant  symbolsji  they  are  only  a  convenient  shorthand  for  differentiating 
the  various  x's  and  z's. 

The  productions  of  the  grammar  are  the  following; 


(1) 

(iji) 

T  .  (i,j  as  in  (ii)  above); 

(2) 

(i>  jjni) 

X.  .--^f(x  .,.„„,x  )  (i,j  as  above,  m  =  1,. . .  .k) ; 

ra  m,l’  ’ 

(3) 

(ifj) 

I.  .->P(y>T); 

-*■ » J 

(4) 

'ij) 

X.  .  z, 

ifO  1,3 

The  trees  generated  by  the  above  grammar  represent  "function-terms" 


in  the  following  way: 
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are  the  initial  functionsj  which  are  fixed  for  the  grammar  in 

questioHo- 

Rule  (2)  is  a  substitution  scheme;, 

Rule  (3)  is  a  d.ef inition-hy-primitive-recursion  scheme;  its  application 
is  meant  to  represent  the  generation  of  a  function  defined  by 

P(0)  ==  y, 

P(n')  =  T(njP(n))s 

where  T  is  a  term  of  the  grammar  containing  only  two  variables  from  the  set 
of  z ' s j  which  has  already  been  generated  by  a  treSo 
The  z's  represent  numerical  vaniableso 

In  view  of  the  above  naive  description;,  the  following  rigorous  definition 
is  naturals 

Definition  3 do  A  function  f  (in  the  naive  sense)  is  representable  in  the 
above  grammarj  with  respect  to  a  system  IT  of  semantic  rules,  iff  there 
exists  a  term  generated  by  the  grammar  whose  interpretation  by  IT  is  identical 
(extensionally)  with  fo 

Note  that  the  representation  of  a  fiinction  in  the  above  sense  is  not 
unique,  since  it  depends  upon  the  generation  tree  considered. 

Theorem  3.1o  There  exist  sets  f^,o..,f  such  that  for  a  suitable  system  JJ 

- - -  -  I  m  - 

all  the  primitive  recursive  functions  are  representable  in  the  grammar. 

Proof.  The  assertion  is  a  direct  consequence  of  a  result  of  R.M.  Robinson  [iS^, 
which  shows  that  in  order  to  obtain  the  primitive  recursive  functions  in  the 
above  manner  it  is  sufficient  to  take  k  =  3,  n^  =  n^  =  1,  n^  =  2.  (Cf. 
Robinson's  paper  for  the  various  possible  choices  of  initial  functions.) 


21  - 


Remark.  A  similar  result,  due  to  J.  Rotinson  [l?]?  dints  at  the  possihilitjr 
of  obtaining  all  the  (general)  recursive  functions,  if  the  initial  functions 
are  supplemented  hy  certain  transformations,  in  a  certain  sense  (such  as 
minimization,  inversion,  etc.);  i.e.,  it  may  he  possible  to  obtain 
representations  of  all  the  general  reoursive  functions  by  means  of  a  transformation 
grammar,  the  representations  of  the  primitive  recursive  functions  serving  as 
kernel. 


A  generation  rule  P  is  applied  on  a  branch  B  of  a  tree  j"*  iff  there 
is  a  node  on  B  such  that  its  direct  successors  in  P  are  obtained  by  P 
from  that  node,  (Strictly  speaking,  this  definition  should  be  phrased  in  terms 
of  the  symbols  labelling  the  nodes,) 

Definition  3,2,  The  degree  of  substitution,  R(f)>  of  a  function-term  f, 
given  as  a  generation  tree,  is  the  maximum  number  of  applications  of  rules 
of  type  (2)  (i,j,m)  on  a  branch  of  the  tree. 

Definition  3,3.  The  degree  of  recursivity .  l(f),  of  a  function-term  is 
defined  in  a  way  similar  to  R(f),  with  regard  to  applications  of  rules  of 
type  (3)  (i,j). 

Example.  The  degree  of  substitution  of  the  following  tree  is  2: 

T 


f2(x2^1,,,,,X2^1o) 


^  22 


The  term  generated  hy  the  above  tree  is 

°  °  ^^2s)2^ °  °  *  ^^2plO^  ° 

It  is  evident  that  these  degrees  are  measures  of  the  complexity,  in  certain 
sensesp  of  functions,  in  a  suitable  interpretation.  The  degree  of  substitution, 
for  instance,  is  a  measure  of  the  need  to  employ  substitutions  in  order  to 
define  a  function  in  the  above  manner<, 


A  further,  more  specific,  exjilication  of  the  concept  is  given  by  the 
following  definitions 


Definition  3o4o  Let  f  be  a  function  (in  the  naive  sense)  which  is  representable 
with  respect  to  the  semantic  r'oles  FT  .  If  is  the  set  of  terms 

representing  f,  then  the  degree  of  substitution  of  f  is 

S(f)  =  min|-i^  pH(  P)  „ 


The  degree  of  recursivity  of  a  function  is  defined  in  a  similar  way. 


Remark.  Different  explications  of  the  above  concepts  are  obtained  if  in 
Definitions  3o2  and  3 <.3  the  degree  is  defined  as  the  maximum  number  of 
'applications  of  the  same  rule  (of  the  appropriate  type)  on  a  branch  of  the  tree. 
The  difference  between  the  two  explications  is  analogous  to  the  difference 
between  degree  of  nesting  aind  degree  of  self-embedding  (of.  section  1),  and 

t 

vrill  be  clarified  in  the  sequel. 

Theorem  3o2.  If  the  degree  of  nesting  of  the  tree  with  respect  to  rules 

(n) 

of  type  (2)  (i,j,m)  is  denoted  by  y'  '^(r'),  and  the  degree  of  nesting  with 

'then  R(r)  = 


respect  to  rules  of  type  (3)  (i,j)  ^ 


^  23 


and  l(r’)  =  V^'^^(r)o  Moreover,  if  £(P)  denotes  the  degree  of  self- 
embedding  with  respect  to  the  symbol  T  only?  then  l(P)  =£(P)o 

Proof.  Obvious. 

Remark.  For  the  explications  mentioned  in  the  Remark  following  Definition  3.4j 
the  first  assertion  of  the  above  theorem  is  true  for  the  degree  of  self¬ 
embedding  of  the  tree.  In  this  case,  it  is  clear  why  the  degree  of  nesting 
is  to  be  preferred  to  the  degree  of  self-embedding  as  an  explication  of  complexity. 

The  reduction  of  the  degrees  defined,  in  Definition  3.4  to  the  concepts 
defined  in  Section  1  will  be  fully  utilized  in  the  proof  of  Theorem  3.4  below. 

Theorem  3.3.  A  set  of  tapes  generated  (or;  accepted)  by  a  finite  automaton 
is  a  primitive  recursive  set  (via  an  appropriate  Godel-numbering) . 

Proof.  The  proof  is  obtained  by  means  of  slight  alterations  in  the  proof  of 
the  main  theorem  of  Myhill  [l5j  Chapter  4] >  according  to  which  every  set  of 
tapes  accepted  by  a  linear  bounded  automaton  is  primitive  recursive. 

Since  every  finite  automaton  is  a  fortiori  a  linear  bounded  automaton, 
the  present  theorem  may  be  proved  using  a  theorem  of  Ritchie  [l6,  p.  76^ 
according  to  which  the  set  of  functions  computed  by  a  linear  bounded  automaton 
is  the  smallest  set  which  contains  the  constant  functions,  the  successor  and 
multiplication  functions,  and  is  closed  und,er  explicit  transformations,  composition 
and  limited  recursion. 

Theorem  3.4<»  Pet  TT  be  a  system  of  semantic  rules  with  respect  to  which 
all  the  primitive  recursive  functions  are  representable  in  the  grammar.  Then, 
for  every  n  and  m,  there  exists  a  primitive  recursive  function  f  such 

- li.  -  - i - 
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that  R(f  )>m  and  l(f  )>no 
-  '  -  man' 

loeoo  there  exists  a  tree  P  such  that  R(  P  )  >m,  l(  P  )>n 

- -  iTijn - mjin  myn 

and  for  every  tree  P  which  represents  the  same  function  (extensionally) ^ 
H(r)^R(  P^  J>  m  ^  r;  „)>n„ 

m  ^  iX  In^  H 

Proof.  .  It  is  clearj  on  the  basis  of  a  theorem  of  Chomsky  [SjTJs  that  the  set 
of  trees  r  of  the  grammar,  for  which  R(P)5cm  or  1(1"*  )^n,  may  he  generated 
by  a  finite  automaton?  thus,  hy  Theorem  3.3?  the  enumerating  function  of  the 
set  of  these  trees,  which  is  defined  hy  the  automaton,  is  primitive  recursive, 

I 

and  thus,  the  diagonal  function  f  defined  with  the  aid  of  the  enumerating 

function  is  primitive  recursive.  Define  g(n)  =  f(n)+l.  Then  g(n)  is  also 

primitive  recursive  and  is  thus,  hy  hypothesis,  representable  with  respect  to  TT  , 

For  any  tree  P  representing  this  function,  it  is  clear  that  R(P)>  m 

and  l(P)>no  Denote  hy  E  the  set  of  these  trees.  Let  E  he  the  set 

R 

of  trees-  p'  in  S  such  that  R(r’)  =  mini"  ^  j,R  ( P )  5  let  E^  he  the  set 

of  trees  P'  in  E  such  that  l(r*)  =  min-.,„  ^(P)'  I’*"*  is  obvious  that 

R  rcEj^ 

aiiy  member  of  E.^^  may  serve  as  the  tree  P  whose  existence  is  postulated 
IR  m,n 


hy  the  theorem. 
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